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Introduction 



Graduate schools everywhere are faced with the naggingly persistent 
responsibility of selecting from a list of applicants those most likely to 
perform successfully in their programs. The process by which students are 
selected has developed into a modest science falling somewhere between 
running a longshoreman’s morning line-up and the choosing of astronauts. 

The problem has generated a literature which is reviewed here to 
determine what predictors and what criteria are used for graduate student 
selection and to evaluate the relative success of the predictors used. 

The bulk of the research involves correlation analysis , and it is hoped 
that the tabular presentation of these data will provide the reader a 
holistic impression of the varied findings. 

The article is organized into the following five segments: The 

Criterion Problem, Kinds of Predictors, Aptitude Variables as Predictors, 
Environmental Variables as Predictors, and Personality Variables as 
Predictors. 

The Criterion Problem 

The most frequently used criterion is graduate grade point average 
(GGPA) , probably because it is an easy one to use. Systems exist for 
grading student performances, gathering these data, and reducing them to 
one simple statistic — the grade average. There have been sufficient 
questions about grading systems and the concept of grades generally (cf. 
Newsweek, and Payne, 1968) to cast some doubt on this criterion, however. 
Upward creeping college grade averages have magnified these doubts in 
recent years. A 1974 survey of 197 U.S. colleges and universities found 
the average increase in grade average had been .404, or nearly half a 
letter grade, between the years 1960 and 1973 (The Chronicle of Higher 
Education, p. 2). When college grades themselves are cast in the role of 



a predictor their performance has been mediocre to poor (cf. Hoyt, 1965). 

In some cases criteria similar to grade point average are used, e_.£. , 
achievement examination scores or proportions of "A" grades. Other criteria 
for graduate student performance include: (1) success or failure in com- 

pleting an academic program, £.£. , M.S. or Ph.D. (see references in Table 
III); (2) faculty ratings of students other than by grades (cf. Hilton, 
Kendall, and Sprecher, 1970); and (3) self ratings (cf. Hackman, et al. , 
1970). However, the criterion most often encountered in the literature 
was the grade average. 

Kinds of Predictors 

Measures of academic aptitude are some of the most often used 
predictors. These include the Graduate Record Examination Aptitude test 
which measures quantitative (GRE-Q) and verbal (GRE-V) aptitude. The 
Graduate Record Examination Advanced (GRE-A) tests examine knowledge in 
various academic disciplines. The literature reveals The Miller Analogy 
Test (MAT) to be popular among psychology departments and schools of 
education. This paper will concentrate on these test measures (GRE-V, 

GRE-Q, GRE-A, and MAT) and undergraduate grade point average (UGPA) , because 
their popularity makes possible a comparative analysis. UGPA may be said to 
represent an intelligence measure, but as a sample of past performance it 
also undoubtedly reflects motivational and other individual differences 
(Tyler, 1965, p. 108ff.). 

Other predictors we encountered in our review are myriad. Here are 

some : 

The number of courses taken in a specific discipline. 

The grades in specific courses. 
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Written statements by the candidate. 

Letters of recommendation. 

Personal interviews. 

Biographical data (from birth order to age, to amount of 
laboratory experience, etc.) 

Quality of undergraduate institution. 

Environments of undergraduate institutions. 

Personality and interest measures. 

Measures of motivation. 

We see here most of the sorts of predictors used in any selection 
process. The predictors reported in the open literature are predominantly 
individual intelligence measures. The results obtained using such measures 
will be presented in the next section of this paper* 

Aptitude Variables as Predictors 

By far the most popular method of analyzing the relationship between 
predictors and criteria is via correlational statistics, e_._g_. , product -moment 
and bi-serial. 

Tables I, II, and III summarize data from 33 analyses made during the 
past decade. The coefficients are presented without referring to their 
statistical significance. The reader should also be aware of the fact that 
not all of these correlations represent the results of crossvalidation 
efforts . 

The studies in the literature using multiple regression analysis are 
not reviewed here because the varying mix of predictors makes the findings 
difficult to compare. Generally speaking, the inclusion of several pertinent 
variables in a multiple regression analysis can, of course, improve the 
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correlation. Other studies using non-comparable analytical techniques, 
e.j». , discriminant function analysis, have not been included. Table I 
presents data from studies utilizing either graduate grade point average 
or achievement examination scores as the criterion. Table II shows studies 
using faculty ratings of students as criteria and in Table III completion 
of degree requirements is the criterion. All three tables present Graduate 
Record Examination- Verbal , Graduate Record Examination-Quantitative, 

Graduate Record Examination-Advanced, the Millers Analogy Test, and Under- 
graduate Grade Point Average as predictors. Studies in academic areas such 
as psychology and education seem to have a disproportionately heavy repre- 
sentation in Tables I-III, due, no doubt, to the popularity of this sort 
of research in these disciplines. A huge proportion of the coefficients 
presented on the tables represent statistically significant relationships; 
however, it is not statistical significance, but, predictive significance 
that we wish to emphasize in this review. Individuals participating in the 
selection of students for graduate study, will, of course, want also to 
consider the magnitudes of the correlation coefficients in the context of 
their departments T selection ratios and current base rates of student success. 
These factors play a major role in determining whether or not a correlation 
between a criterion and a predictor set is "large enough" (Taylor and Russell, 
and Abrahams, Alf and Wolfe). 

Choosing, albeit arbitrarily, a correlation coefficient value of .35 
as indicating marginal predictive respectability we find in Table I that 
less than half of the 42 GRE-V coefficients equal or exceed that value. A 

higher standard, + .50, yields only five correlations. The GRE-Q is even 
less inspiring. Ten of the forty-three coefficients exceed .35, and 
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^Unless otherwise noted the performance measure is graduate grade point average. ^Graduate Record Examination-Verbal. 
^Graduate Record Examination-Quantitative. ^Graduate Record Examination (Verbal score plus Quantitative score). 
^Graduate Record Examination-Advanced. (Where applicable, area is indicated in "Course of Study" column). 

^Miller Analogies Test. 7crade Point Average in undergraduate institution. ^See Reference List for complete reference. 
a Reading Examination t)Compr ehensive Examination 
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Summary of Correlations 
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Graduate Record Examination-Verbal Graduate Record Examination-Quantitative 
'Graduate Record Examination-Total (Verbal score plus Quantitative score) 

Graduate Record Examination-Advanced (where applicable, area indicated in 'course of study" column • 

j 

Undergraduate Grade Point Average See Reference List for complete reference 



four are above .50. The same results exist for the GRE-Advanced data; 
slightly more than half the correlations exceed .35 and three are higher 
than + .50. MAT results are somewhat better, but still modest. Five of 
the nine coefficients are above .35, but only one is above .50. 

In what appears to have been an exhaustive review of studies using 
the GREs , Willingham summarizes the predictive track records of the GREs , 

UGPA, and letters of recommendation. The validities of these measures in 
predicting criteria such as GGPA and overall faculty ratings are presented 
(via median validity coefficients) as are their median validity coefficients 
in each of nine fields of graduate study. These median validity coefficients 
are nearly always less than .40, and have a typical value of around .30. 

One might argue that aptitude measures should not be particularly 
good predictors of scholastic productivity, because student motivation may 
be such an important factor. Undergraduate grade average should reflect 
both intelligence and motivation and should, therefore, be a better predictor 
of graduate performance than aptitude measures alone. Right? Wrong! Table 
I shows only one study in which the correlation between GGPA and UGPA was 
above .35. None exceeded .50. 

These results do not produce great confidence in what one might 
otherwise believe to be useful predictors. Perhaps it is the criterion, 
graduate grades, that is responsible for the relatively low correlations. 
Tables II and III present alternative criteria - but with the same kind of 
indifferent results. Using completion of degree requirements as the standard, 
we find the correlations to be positive, but again at a very modest level. 

Of the 53 correlations with GRE-V, GRE-Q, GRE-A, MAT, and UGPA, only one 
exceeds .50, and it’s the same one that exceeds .35. 
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When we look at the faculty evaluation criterion, only two (GRE-A and 
UGPA) of the 19 coefficients exceed .35, and none is over .50. Why do these 
predictors perform so modestly? 

Somewhat higher validity coefficients were found in studies carried 
out at the Naval Postgraduate School in 1966 and 1967. Here we find cor- 
relations between grades and GRE-V of .51, .44 and .43, and GRE-Q correlations 
ire even higher, .73, .70 and .65. These higher correlations may be caused by 
such factors as adequate financial support of the students during their 
studies. They may also reflect a homogeneous level of motivation for older, 
more career ensconced students. Other plausible explanations are statistical. 
In nearly all the studies reported hers, with the exception of the Naval 
Postgraduate School studies, the predictor data were used to select students 
for the graduate programs. Truncated samples undoubtedly resulted, with 
only the higher ends of the predictor distributions being represented. This 
being the case, the correlations would be lower than they would have been 
without the restriction of range on tn* .redial is, t Thorndike, 1949, p. 170) 
Further, as Platz, et al. , Strand, and Dawes (1975) have shown, if an accepted 
applicant is low in one of the attributes used in the selection process, 
nmre is an exceedingly high probability thdt he wil .1 be high in others. 

T Lerefore, if a student’s entrance examination scores are low, it is probable 
that his undergraduate grades will be high. The correlations between grades 
and entrance exam scores for all accepted students T~ill be low. Added to 
these problems is the inexact nature of most of the criterion measures and 
the restricted range of GRE scores with which most schools work. As a 
result, che low correlations between test s^or^s and grades can almost be 
expected . 
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Of course, the intellectual factors represented in this section are 
not the only ones which affect student performance. The low correlations 
may reflect the influence of variables other than intelligence. In the 
Predictors section several such factors were listed. An important amount of 
research has been performed in two of them, the college environment and 
individual personality. Though the major part of this research has focused 
on undergraduates, the findings might well be extrapolated to graduate level 
performance prediction. A discussion of some of these studies is included 
in the following two sections. 

Environmental Variables as Predictors 

Most of the research into college environment and institutional quality 
revolves around Astin' s examination of the subject during the 1960 f s. Astin 
and Holland developed an Environmental Assessment Technique. This includes 
three sets of variables: (1) the six Holland (Holland, 1959) vocational 

classifications (Realistic, Intellectual, Social, Conventional, Enterprising 
and Artistic), (2) the size of the institution, and (3) the intelligence 
level of the student body. This latter is an estimate derived from a sample 
of undergraduates entering 335 institutions. The estimate was based on their 
National Merit Scholarship scores (Astin and Holland, 1969, p. 308). Astin 
found the environmental variables. Intellectual, Enterprising and Artistic, 
to correlate positively with student body intelligence level. The Realistic, 
Social and Conventional orientations correlated negatively. Astin found that 
the aspiration of talented students to obtain a Ph.D. was negatively affected 
by the size of the student body and the Conventional orientation (Astin, 1963). 
Institutions scoring high on these factors tended to emphasize sports and 
social activities at the expense of scholarship. He also found that student 
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faculty relations were less effective and improvement in study habits was 
inhibited. In another study, Astin found that student achievement as 
measured by GRE advanced tests in the areas of social science, natural science 
and humanities, correlated poorly with traditional indices of institutional 
quality such as intellectual level of classmates, competitiveness or insti- 
tutional affluence (Astin, 1968) . Institutions characterized by intelligent 
students, competitiveness and affluence do turn out students that perform 
better on a variety of accomplishment measures. This difference disappears, 
however, when a correction for individual differences in student ability is 
made. Freshman grade point averages were higher for students in the more 
selective schools, but when selectivity was taken into account, such factors 
as size, wealth, location, type of control (e_.£. , state, private, religious) 
and curriculum appeared to make little difference in student performance 
(Astin, 1971, p. 27). 

Hood and Swanson (1965) using somewhat different methods characterized 
colleges in the state of Minnesota as agriculture, institute of technology, 
college of liberal arts (all in the University of Minnesota) , private liberal 
arts, Catholic male, Catholic female, state and junior. They were able to 
state that a student falling at the 50th percentile in the Minnesota Scholastic 
Aptitude Test would probably be expected to fail at the University Institute 
of Technology or College of Liberal Arts with a 1.4 or 1.6 (out of 4.0) average, 
be placed on probation with a 1.9 at a typical private liberal arts college 
or Catholic men’s college, and to make a C+ average at a Catholic women’s 
college, a junior college or a state college. 

Hackman, Wiggins and Bass (1970) found that the "quality" of the 
undergraduate institution, as assessed by members of the University of 
Illinois Psychology Department faculty, correlated .30 with the student’s 
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own assessment of his progress toward a Ph.D. , .31 with faculty judgment and 
.43 with his relative "success"^ - six years out of school. 

There is some indication that newer environmental assessment techniques 
which identify a school's "personality" may provide insight into the perform- 
ances of its graduates in advanced degree programs. In summary, it would 
seem that use of a measure of undergraduate institutions' selectivity, like 
Astin's seven point scale (Astln, 1971, p. 48), should help improve predictions 
of graduate study performance. 

Personality Variables as Predictors 

Personality factors would seem to account for some of the variance 
unexplained by academic aptitude measures and undergraduate grades. One 
variable would be the need for achievement. Projective testing of this 
motivation orientation has been carried out at Harvard by McClelland and his 
colleagues. Early results were spotty, varying between moderate positive to 
negative correlations between Themantic Apperception Test (TAT) need achieve- 
ment scores and college grades (McClelland, Atkinson, Clark and Lowell, 1953). 
Personality inventory measurements of achievement motivation have fared 
little better. A study by Gough and Hall (1964) found no significant 
correlations between the two achievement scales on the California Personality 
Inventory (CPI) and medical school GPA, but significant correlations were 
found for sociability (.35), tolerance (.34) and intellectual efficiency 
(.40). 
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Success" ranged from failure-to-complete-the-doctorate 
to a position in a "highly prestigious" institution. 



to appointment 
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An extensive study of high aptitude students by Holland and Astin (1962) 
found the following CPI scales to correlate significantly with college grade 
point average: For men, femininity (.26), social presence (-.26), responsi- 

bility (.16), socialization (.22), self-control (.15), sociability (-.21), 
capacity for status (-.19), self-acceptance (-.15) and flexibility (-.10); 

For women, achievement via conformance (.28), social presence (-.19), 
responsibility (.25), femininity (.19), socialization (.22), self-control 
(.22), achievement via independence (.18) and self-acceptance (-.15). 

In an unpublished study at the Naval Postgraduate School, Senger, 

Wyatt and Knapp found a statistically significant correlation (.26) between 
CPI achievement via independence and grades, and also correlations between 
intellectual efficiency (.25), psychological mindedness (.24) and flexibility 
(.17) and grades. 

In Watson’s investigation of achievement predictors and maladjusted 
and normal college males, six significant relationships (p<.05) between CPI 
variables and grades were found among maladjusted upperclassmen: capacity 

for status (.30), well-being (.22), tolerance (.27), achievement via con- 
formity (.24), achievement via independence (.41) and intellectual efficiency 
(.32). Among maladjusted freshmen, six of the correlations were significant: 
dominance (.30), self-acceptance (.22), tolerance (.21), achievement via 
independence (.38), intellectual efficiency (.27) and psychological mindedness 
(.33). There were rio significant correlations between the CPI variables and 
GPA for the normals. 

Evans found no significant correlations between the GPA of a sample 
(N=102) of Geneva college freshmen and the CPI scales for achievement via 
independence, achievement via conformity and intellectual efficiency. 
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The Achievement scale on the Edwards Personal Preference Schedule 



(EPPS) is another popular measure of this motivation, but, as in the case 
of the CPI, the literature relating it to academic performance seems sur- 
prisingly thin. At Carnegie Institute of Technology, Krug (1959) found 
significantly higher scores on the Achievement scale among academic over- 
achievers as compared to under-achievers. Gabhart and Hoyt (1958) found 
similar results at Kansas State. Both studies found the Need for Order 
discriminated between under- and over-achievers. At the Naval Postgraduate 
School an unpublished study by Golanka and Gilmore (1967) found under-achievers 
scoring significantly higher than over-achievers on Achievement and Order. 
Senger, Wyatt and Knapp (1969) found, in another study at the Naval Postgraduate 
School, a significant positive correlation (.23) between EPPS Achievement and 
graduate grade point average. A .10 correlation between Iowa State engineering 
student overall GPA and the EPPS achievement scale and a -.24 (p=.01) cor- 
relation with the EPPS affiliation scale was found by Muchinsky and Hoyt. 

In sum, the situation still seems as described in 1949 (Donahue, Coombs and 
Travers) : motivational measures and grades tend to be only slightly 

interrelated . 

Studies relating Strong Vocational Interest Blank (SVIB) to academic 
performance are examples of attempts to relate scholarship to personal 
interests. Hauntras, Lee and Hebahlran (1973) found the SVIB Academic 
Achievement (AACH) scale to correlate .37 with grades for 423 freshmen. 

Johnson (1969) found correlations between AACH and GPA to be .17 for arts and 
science students and .02 for business administration students. In the 
original validation studies, Campbell and Johansson (1966) found a correlation 
of .36 between AACH and GPA for their freshmen cross-validation group. Lindsay 
and Althouse (1969) found an AACH-GPA correlation of .10 for male freshmen and 
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of .25 for women. A .35 correlation between AACH and GPA was found by 
Wagman (1971) for an undergraduate and graduate sample. Results of a study 
by Frank among a sample of Berkeley students indicate a correlation between 
the AACH scale and first-year GPA to be .17, .06 with third-year grades and 
.13 with fourth-year performance. 

Scores on the SVIB occupational scales are predictive of students 1 
tendencies to stick with a curriculum, but usually are not predictive of 
grades (Kellogg, 1968). The lack of a correlation between SVIB scores and 
grades may be another example of the impact of restriction of range, jL.e^. , 
self selection may have yielded groups having homogeneous SVIB occupational 
scale scores. 

Summary and Conclusions 

Examination of the accompanying tables of correlations between 
intellectual and other measures and criteria of graduate student performance 
does not encourage one to increase his faith in the validity of the popular 
predictors used singly. It must be stated, however, that the relationships 
are positive, and when we take into account that most are based upon truncated 
samples not including low scorers, admission of students with low test scores 
and high grades, and problematic criteria, validities of these measures may 
be better than the typical study makes them appear. 

The phenomenon of low predictor - criterion relationships may not be 
restricted to academic performance. Ghiselli (1966) did not find very high 
coefficients (almost always less that +.30 with performance criteria) in his 
survey of the validity of occupational aptitude tests. Difficulties in 
predicting performance appear to be universal. It should be noted again that 
the studies reviewed here were often of concurrent, rather than predictive. 
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validity design. The correlations are, therefore, probably smaller than they 
would have been without restriction of range. It should be further cautioned 
that the tables include findings which have not been cross-validated, probably 
exaggerating the strengths of the relationships presented. 

Can measures of non-intellectual factors be used to improve predictions? 
It appears that the "quality 11 of the undergraduate institution may be useful 
in predicting graduate student performance. The underlying factor here may 
be the selectivity of the college; in any case, this "environmental" input 
may be useful in interpreting the meaningfulness of the undergraduate grade 
point average. 

Itwuld seem that standard measures of motivation and interest should 
be worthwhile supplemental predictors; unfortunately, studies to date do not 
support this expectation. Investigations using the TAT, CPI and EPPS do not 

provide much data which would give one confidence in finding a useful measure 

of motivation. The investigations using these instruments show their predictive 
power is usually low. The SVIB offers little more; the few studies show neither 
the occupational scales nor the Academic Achievement scale offering strong 
relationships with academic performance. Perhaps Tyler (1965, p. 119) provided 
an explanation when she wrote: 

A conclusion suggested by this research and compatible 
with all the previous work in this and other settings is that 
the differences in motivation leading to differences in school 
achievement are not those that personality theorists, with 

their background in the clinic and the hospital, tend to think 

of first. They are not differences in basic drives but in 
learned habits of work. They are not differences in the degree 
to which negative qualities like anxiety and neurotic traits are 
present but rather the degree to which strong and well organized 
positive qualities such as interests, commitment, or enthusiasm 
about some line of endeavor characterize an individual. 

The necessity to choose from among the applicants to graduate schools 

persists, however, and though the relationships between single predictors and 
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criteria are not particularly strong, they can be useful for decision making... 
if selection ratios are sufficiently small and prior base rates of success 
are auspicious (Meehl and Rosen, 1955). Dawes (1971, p. 180) stated that 
top graduate departments were considering as many as 100 applicants for 
every graduate student selected for admission. With such a selection ratio, 
even predictors with low validities can be expected to be useful (Taylor and 
Russell, 1939, and Abrahams, Alf and Wolfe, 1971) in decision making. So, 
from a decision making point of view, the predictors of graduate school 
performance may be, in the current marketplace, good enough. 

Finally, the reader should be reminded that studies using multiple 
predictors simultaneously were not reviewed in this paper. Such studies, 
usually using multiple regression, often yield moderately higher validity 
coefficients than are found when using a single predictor — especially when 
one of the predictors functions as a suppressor variable (Weitzman, pp. 236- 
238). 
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